-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use chunking in web upload #26306
Use chunking in web upload #26306
Conversation
@PVince81, thanks for your PR! By analyzing the history of the files in this pull request, we identified @butonic, @luckydonald and @DeepDiver1975 to be potential reviewers. |
Important notejquery blueimp doesn't support parallel uploading of chunks, so the chunks are always uploaded sequentially. This isn't too bad as it still brings the benefit of avoiding timeouts. |
tested - works 👍 |
|
Hmm this is tricky because if you test it by making the server unavailable, there are other ajax calls (notifications, getstoragestats) that will fail too. I tried disabling the reload and it seems that jquery.fileupload doesn't automatically resume. |
@PVince81 I need confirmation that this will work in IE11. How could I verify this asap? |
When the connection is closed the Response status is 0, which triggers the reload. Reloading on Status = 0 was introduced with #16783 (comment) because of a missing Access-Control-Allow-Origin header on the IdP. The proper fix is to make the IdP send this header. While we currently cannot detect an abort we might ba able to configure mod_shib to send 403 forbidden for xhr requests. That is what mod_mellon does: https://github.com/UNINETT/mod_auth_mellon/blob/master/README#L169-L173 Either test mod_mellon or try to make apache send 403 responses when no shibboleth session is present for xhr requests (X-Request-With: XMLHttpRequest). The latter might be very tricky ... see https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS |
That being said ... I can live with the page refreshing too often... I cannot live with IE11 failing to upload files >2.2GB. |
Upload with 32-bit IE11 worked well (In my example 2.281.105.750 bytes). Progress bar shows very wrong estimated values in the range from days to seconds. It took some time after the progress bar disappeared and the file appeared. |
✔️ with 4.6GB on IE11 🎉 @PVince81 is awesome! |
There's a potential bug, see the discussion here owncloud/client#5476 (comment). Need to make sure that the uploaded chunk names are ascii-sortable in the correct order to avoid corruption because the endpoint expects this. |
|
|
|
At this stage no progress can be shown because the transfer already ended. So one idea would be to already insert the entry into the file list but let it appear with a spinner. |
If we had a good solution to get rid of the IDP "status=0" fail detection it would be great, because that check is messing up with more stuff. #26306 (comment) |
Will need further work there. |
this is a major blocker. Need to talk with @DeepDiver1975 how to redesign the DAV endpoints to allow public access |
Let's finish this without the public link part then. |
Any blockers? Open separat issue for discussion on public link? |
No blocker, just need to find time to finish this... The biggest missing bit now is error handling in case of abort. |
@PVince81 Thanks in advance and 👍 |
bcc901d
to
caba170
Compare
Because some people will want to tweak it and changing a hard-coded value would cause integrity check errors.
Giving up on trying to make it retry chunks. This jquery.fileupload lib is really painful to use. But for now let's focus on getting this merged. The main purpose for now is to make it possible to upload files bigger than 2 GB in IE11. |
This would also require hacking around jquery.fileupload since that one thinks we're already done since the last chunk was uploaded already... The lib isn't really compatible, too much hacking... Maybe one day we should find another lib or develop our own. |
Jenkins is kaputt:
Resent PR separately: #28415 |
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Description
Switch on chunking for files bigger than 10 MB.
Some additional fixes to make it use the correct URL/client.
Related Issue
Fixes #8955
Motivation and Context
Makes it possible to avoid timeouts when uploading bigger files.
Might also make it resumable (still to be tested)
How Has This Been Tested?
For all tests, check the network console:
Screenshots (if appropriate):
Types of changes
Checklist: